human editor
WINELL: Wikipedia Never-Ending Updating with LLM Agents
Reddy, Revanth Gangi, Dixit, Tanay, Qin, Jiaxin, Qian, Cheng, Lee, Daniel, Han, Jiawei, Small, Kevin, Fan, Xing, Sarikaya, Ruhi, Ji, Heng
Wikipedia, a vast and continuously consulted knowledge base, faces significant challenges in maintaining up-to-date content due to its reliance on manual human editors. Inspired by the vision of continuous knowledge acquisition in NELL and fueled by advances in LLM-based agents, this paper introduces WiNELL, an agentic framework for continuously updating Wikipedia articles. Our approach employs a multi-agent framework to aggregate online information, select new and important knowledge for a target entity in Wikipedia, and then generate precise edit suggestions for human review. Our fine-grained editing models, trained on Wikipedia's extensive history of human edits, enable incorporating updates in a manner consistent with human editing behavior. Our editor models outperform both open-source instruction-following baselines and closed-source LLMs (e.g., GPT-4o) in key information coverage and editing efficiency. End-to-end evaluation on high-activity Wikipedia pages demonstrates WiNELL's ability to identify and suggest timely factual updates. This opens up a promising research direction in LLM agents for automatically updating knowledge bases in a never-ending fashion.
Automated Meta Prompt Engineering for Alignment with the Theory of Mind
Baughman, Aaron, Agarwal, Rahul, Morales, Eduardo, Akay, Gozde
We introduce a method of meta-prompting that jointly produces fluent text for complex tasks while optimizing the similarity of neural states between a human's mental expectation and a Large Language Model's (LLM) neural processing. A technique of agentic reinforcement learning is applied, in which an LLM as a Judge (LLMaaJ) teaches another LLM, through in-context learning, how to produce content by interpreting the intended and unintended generated text traits. To measure human mental beliefs around content production, users modify long form AI-generated text articles before publication at the US Open 2024 tennis Grand Slam. Now, an LLMaaJ can solve the Theory of Mind (ToM) alignment problem by anticipating and including human edits within the creation of text from an LLM. Throughout experimentation and by interpreting the results of a live production system, the expectations of human content reviewers had 100% of alignment with AI 53.8% of the time with an average iteration count of 4.38. The geometric interpretation of content traits such as factualness, novelty, repetitiveness, and relevancy over a Hilbert vector space combines spatial volume (all trait importance) with vertices alignment (individual trait relevance) enabled the LLMaaJ to optimize on Human ToM. This resulted in an increase in content quality by extending the coverage of tennis action. Our work that was deployed at the US Open 2024 has been used across other live events within sports and entertainment.
'We need to set the terms or we're all screwed': how newsrooms are tackling AI's uncertainties and opportunities
In early March, a job advert was doing the rounds among sports journalists. It was for an "AI-assisted sports reporter" at USA Today's publisher, Gannett. It was billed as a role at the "forefront of a new era in journalism", but came with a caveat: "This is not a beat-reporting position and does not require travel or face-to-face interviews." The dark humour was summed up by football commentator, Gary Taphouse: "It was fun while it lasted." As the relentless march of artificial intelligence continues, newsrooms are wrestling with the threats and opportunities the technology creates.
Edisum: Summarizing and Explaining Wikipedia Edits at Scale
ล akota, Marija, Johnson, Isaac, Feng, Guosheng, West, Robert
An edit summary is a succinct comment written by a Wikipedia editor explaining the nature of, and reasons for, an edit to a Wikipedia page. Edit summaries are crucial for maintaining the encyclopedia: they are the first thing seen by content moderators and help them decide whether to accept or reject an edit. Additionally, edit summaries constitute a valuable data source for researchers. Unfortunately, as we show, for many edits, summaries are either missing or incomplete. To overcome this problem and help editors write useful edit summaries, we propose a model for recommending edit summaries generated by a language model trained to produce good edit summaries given the representation of an edit diff. This is a challenging task for multiple reasons, including mixed-quality training data, the need to understand not only what was changed in the article but also why it was changed, and efficiency requirements imposed by the scale of Wikipedia. We address these challenges by curating a mix of human and synthetically generated training data and fine-tuning a generative language model sufficiently small to be used on Wikipedia at scale. Our model performs on par with human editors. Commercial large language models are able to solve this task better than human editors, but would be too expensive to run on Wikipedia at scale. More broadly, this paper showcases how language modeling technology can be used to support humans in maintaining one of the largest and most visible projects on the Web.
Wikipedia Will Survive A.I.
Welcome to Source Notes, a Future Tense column about the internet's information ecosystem. Wikipedia is, to date, the largest and most-read reference work in human history. But the editors who update and maintain Wikipedia are certainly not complacent about its place as the preeminent information resource, and are worried about how it might be displaced by generative A.I. At last week's Wikimania, the site's annual user conference, one of the sessions was "ChatGPT vs. WikiGPT," and a panelist at the event mentioned that rather than visiting Wikipedia, people seem to being going to ChatGPT for their information needs. Veteran Wikipedians have couched ChatGPT as an existential threat, predicting that A.I. chatbots will supplant Wikipedia in the same way that Wikipedia infamously dethroned Encyclopedia Britannica back in 2005.
Google search responds to BankRate, more brands using AI to write content
Artificial intelligence (AI) has been a scorching hot topic lately, especially since the launch of ChatGPT Nov. 30. Microsoft Bing has plans to add ChatGPT to search. Some have questioned whether it's a Google killer. Bankrate is the latest example. It is having some of its content written by machines but reviewed by human editors.
Users trust AI as much as humans for flagging problematic content
Social media users may trust artificial intelligence (AI) as much as human editors to flag hate speech and harmful content, according to researchers at Penn State. The researchers said that when users think about positive attributes of machines, like their accuracy and objectivity, they show more faith in AI. However, if users are reminded about the inability of machines to make subjective decisions, their trust is lower. The findings may help developers design better AI-powered content curation systems that can handle the large amounts of information currently being generated while avoiding the perception that the material has been censored, or inaccurately classified, said S. Shyam Sundar, James P. Jimirro Professor of Media Effects in the Donald P. Bellisario College of Communications and co-director of the Media Effects Research Laboratory. "There's this dire need for content moderation on social media and more generally, online media," said Sundar, who is also an affiliate of Penn State's Institute for Computational and Data Sciences.
How Artificial Intelligence is Transforming Video Editing?
Artificial intelligence's development has altered many facets of our existence. Video editing software is one example of a shift. When it comes to operating simple applications like video editing software, AI technology is starting to engage and even replace workers in some situations. The fundamental notion is that AI will learn from its mistakes and make more precise choices than a human ever could. For individuals who rely on their employment as video editors, this may seem frightening, but there are still numerous ways humans can help in the management that cannot be handled by an algorithm just yet.
Is Artificial Intelligence Racism Proof?
Artificial Intelligence is often interchanged with the word robotics. Although AI might be the single most tremendous technology revolution of our days, with the potential to disrupt almost all aspects of human existence, it does not mean robots are not crucial in our digital world. From helping fight the recent COVID-19 by carrying infectious samples, medicines, food from one place to other to disinfecting public space, to helping manufacturing sectors in assembly lines to inspecting raw materials, robots are almost omnipresent. However, it is still far from being the silver bullet due to the bias of Artificial Intelligence. Recently in May, when Microsoft proposed to replace their human editors with an AI, much was at stake due to Microsoft's reputation of racist bot Tay.
Microsoft News just cut dozens of editorial workers as it moves towards a robot-driven system of selecting stories
Microsoft News has shed dozens of editorial workers this past week as it moves to an AI-driven system of picking news and away from human editors for MSN.com, one of the world's biggest news destinations. People close to the situation said the layoffs impacted all its contractors in the US, numbering around 50, all of whom are employed by staffing agencies Aquent and MAQ Consulting. Calls and emails to those agencies seeking comment weren't returned. A Microsoft spokesperson said: "Like all companies, we evaluate our business on a regular basis. This can result in increased investment in some places and, from time to time, re-deployment in others. These decisions are not the result of the current pandemic."